Preprocess data

Define train/val/test sets

Deep learning model

Model architecture and compilation

Callbacks

Fit the model

Model interpretation

GradCAM

Classes with incorrectly classified images from test dataset

Conclusions based on GradCAM

1.

There are few beer labels which are very similar with difference an animal in the middle of the label and name of a beer.

Model does not recognize the animal. It may be caused by it being less distinctive than black letters next to it, which are apparently mainly used for recognition of this particular set of labels.

image-4.png

image-5.png

2.

In example below, class label has reflection in important part of the label and this reflection leads to it not being visible to alrogorith. Both predicted and true labels have the same color. In the place of reflections class 67 has letters denoting beer name.

image-6.png

3.

Here both labels have alike designs (strip at the bottom, similar fonts) and bottles. Maybe if both labels were photographed with better lighting, hence edges would be better visible, model would predict correct classes.

image-7.png

4.

In this case model is totally blind.

image-8.png

Other image of class 90, where bottle is not rotated and lighting is different. image-9.png

Conclusion

Designed model with accuracy almost 92% on dataset. The most wrong predictions in one class was for labels from "Browar Za Miastem" brewery (animal in the middle).

Best way to improve the model and make is more uniform (to be usable in images from different distributions) would be implementing data augmentation.

It would be highly recommended to check if random_contrast may make possible detection of the animal in labels mentioned above.